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ABSTRACT 


OBJECTIVE: To determine the effectiveness of Mini-CEX as a formative assessment instrument in 
improving the clinical skills of postgraduate residents in Pediatrics. 

METHODOLOGY: This cross-sectional study was conducted from January to August 2021 at Ziauddin 
Medical University. A total of 176 Mini-CEX encounters were conducted on twenty-two Pediatric 
postgraduate residents by eight faculty members. The non-probability convenience technique was used 
for sampling. Each resident underwent one Mini-CEX encounter per month, and the ratings were 
recorded. An Objectively Structured Clinical Evaluation (OSCE) was conducted before the first Mini-CEX 
session, and the last and the mean scores were compared. Data were analyzed using SPSS version 20. 
ANOVA was used to determine inferential statistics to calculate the p-value and the improvement in 
score over time to report statistical significance. 

RESULTS: Mean scores of all clinical competencies improved with subsequent Mini-CEX sessions. 
Maximum improvement was noted in clinical management skills and organization/efficiency. Marked 
improvement was observed in OSCE scores before the first Mini-CEX session and after the eighth 
session (p-value <0.001). An increase in the satisfaction rate of both faculty and residents with each 
subsequent Mini-CEX session was also noted. 

CONCLUSION: Mini-CEX effectively improves pediatric postgraduate residents’ clinical learning; 
therefore, it should be incorporated into the national residency program. 
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INTRODUCTION 


Mini Clinical Evaluation Exercise (Mini-CEX) is a 
structured and validated workplace-based assessment 
(WPBA) tool, initially developed in 1995 in the USA, to 
evaluate the clinical competencies of internal medicine 
residents’. Three components of Mini-CEX are clinical 
performance, direct Observation, and_ timely, 
constructive feedback. The developed countries have 
widely used Mini-CEX for undergraduate and 
postgraduate training, and _ several international 
research studies have emphasized its feasibility and 
acceptability. However, there are some 
inconsistencies in the research literature regarding its 
effectiveness as a workplace-based assessment 
method’. Moreover, only a few studies from 
developing countries highlight the role of Mini-CEX in 
improving trainees’ clinical skills®. 

Mini-CEX for continuous evaluation of postgraduate 
trainees has not yet been incorporated into the 
pediatric residency program by the College of 
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Physicians and Surgeons Pakistan’. This study was 
designed to determine the effectiveness of Mini-CEX 
as a formative assessment tool in improving the 
learning and _ clinical performance of Pediatric 
postgraduate trainees. The study was conducted to 
evaluate the usefulness of Mini-CEX and make 
recommendations for its introduction as a mandatory 
requirement for the national residency program in 
Pediatrics. Moreover, there is no research study from 
Pakistan on using Mini-CEX as an assessment tool 
for Pediatric residents. Therefore, the effectiveness 
and usefulness of Mini-CEX as a teaching and 
formative assessment tool in the context of Pediatrics 
need to be established, especially in developing 
countries, by conducting research studies on a larger 
scale°. This study aimed to evaluate the role of Mini- 
CEX in improving clinical competencies and skills of 
pediatric residents in the local context and thus will 
serve as a basis for future research. 


METHODOLOGY 


This prospective, cross-sectional study was 
undertaken at the Department of Pediatrics at 
Ziauddin Medical University from January to August 
2021. A total of 22 Pediatric postgraduate residents in 
their second year of training and eight faculty 
members were included in this study. The same 
faculty members conducted all eight Mini-CEX 
encounters of the same group, ensuring that all eight 
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faculty members assessed the postgraduate 
residents. Written informed consent was taken from 
the students and those residents who were 
unwilling.to participate in the study were excluded. 
The hospital's Ethics Review Committee approved the 
study (ERC letter No.2701020SEPED), after which the 
study was initiated. The non-probability convenience 
sampling technique was used for sampling. Before the 
initiation of the study, a workshop to train the faculty in 
Mini-CEX was held. A _ ten-station OSCE was 
conducted before Mini-CEX sessions; another was 
arranged after the last. The same stations were 
included in both OSCE; their mean scores were 
recorded on a performa and compared afterwards. 
Each trainee underwent eight Mini-CEX encounters, 
i.e. one per month. The Mini-CEX encounters were 
assessed, and the scores were recorded on a 
modified Mini-CEX evaluation form, pilot tested before 
being administered. Cronbach's alpha was calculated 
to measure the reliability of items included in the 
clinical encounter form. 

A total of 176 clinical encounters were conducted in 


various clinical settings. The core competencies 
assessed included: physical examination, clinical 
judgment, history taking, clinical management, 


counselling skills, professionalism, and broader 
clinical competence. The performance in Mini-CEX 
encounters was rated on a 9-point Likert scale. If the 
scores were between 1-3, the assessments were 
considered unsatisfactory, i.e. there were gaps in 
knowledge or skills that would not be expected at the 
level of training and some _ concerns about 
professionalism or patient safety. If the trainee was 
generally clinically competent and with clinical 
competencies such as communication skills and 
professionalism at their expected level of training, the 
scores were between 4-6 and the assessments were 
considered satisfactory. If the trainees performed 
beyond their desired level of learning scores with no 
concerns about their clinical competence, they were 
assigned scores between 7-9, considered excellent. 
The satisfaction of assessors and residents with Mini- 
CEX was also recorded on a 9-point Likert scale 
ranging from extremely satisfied to highly dissatisfied. 
Furthermore, the time given to Observation and 
feedback was also recorded. 

Data from Mini-CEX clinical encounter forms were 
entered and analyzed using SPSS version 20. Mean 
and standard deviation (SD) were calculated for 
quantitative variables. ANOVA was used to determine 
inferential statistics to calculate the p-value and the 
improvement in score over time to report statistical 
significance. Paired t-test was applied to compare the 
mean and standard deviation (SD) of students’ OSCE 
scores before and after performing the Mini-CEX. For 
quantitative variables, an independent t-test was 
used. A Chi-square test was used for qualitative 
variables. The results were expressed as percentages 
and frequencies, and a p-value of less than 0.05 was 
considered significant. 
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RESULTS 


A total of 176 Mini-CEX encounters took place over 
eight months. A maximum number of encounters, i.e. 
65(36.9%), were assessed by assistant professors 
(Figure I). As all the faculty members were trained in 
the use of Mini-CEX before the initiation of the study 
to ensure inter-rater reliability, no difference in 
assessment scores concerning faculty designation 
was observed. 

For Mini-CEX encounters, patients with a wide array 
of clinical problems were selected. The majority of the 


clinical cases, i.e. 62(35.2%), belonged to the 
infectious disease category, 31(17.6%) had 
neurological and respiratory problems, and _ the 


gastrointestinal and cardiac cases were 27(15.3%) 
and 25(14.2%), respectively. Of 176 clinical 
encounters, 148(84.1%) were conducted on new 
patients, while 28(15.9%) were on follow-up patients. 
The majority of the cases, i.e. 156(89.9%), were 
moderate in complexity, while 17(9.6 %) were highly 
complex, and only 1(0.6%) was of low complexity. 
Those clinical cases with a straightforward diagnosis 
and no comorbidity were considered low complexity. 
Those with an evolving clinical presentation or one or 
two comorbidities were moderately complex cases. In 
contrast, patients with multi-morbidity and high 
analytical complexity requiring a holistic approach 
were considered highly complex®. A total of 37 
(21.02%) % observations were made in_ the 
emergency department, 33(18.76%) were conducted 
in the out-patient department (OPD), 51(28.97%) in 
the Pediatric ward, 43(24.43%) took place in the 
Pediatric Intensive Care Unit (PICU) and 12(6.82%) in 
the Neonatal Intensive Care Unit (NICU). The mean 
observation time was 16.5 minutes (range 15-20 
minutes), while the mean feedback time was 9 
minutes (range 5-10 minutes). It was observed that 
during May, the examiner satisfaction with Mini-CEX 
decreased, which could be explained by the increased 
clinical workload of the faculty during this month. 
However, an overall increase in the satisfaction rate of 
both trainees and assessors with Mini-CEX 
encounters for eight months was observed (Figure Il 
and Ill, respectively). 

A Cronbach's alpha of 0.9 indicated high internal 
consistency reliability of Mini-CEX items. Mean scores 
and Standard Deviation (SD) of all the competencies 
observed during the eight Mini-CEX sessions (Table 
I). A progressive improvement in mean scores of 
competencies in subsequent Mini-CEX encounters 
was observed, i.e. from encounter 1st till encounter 
8th; there was a rising trend in the mean scores of the 
trainees (p-value <0.001) (Table II). The mean score 
on the first encounter was highest for professionalism 


and physical examination skills. In contrast, the 
maximum improvement in the mean score was 
observed in clinical management — skills and 


organization/efficiency. The mean score in OSCE Pre 
and post-Mini-CEX sessions is shown in table Il. 
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Paired sample t-test was significant between Figure Ill: Mean Plot of Examiner Satisfaction 
the mean scores of pre and post-Mini-CEX OSCE using MINI CEX 
(Table Ill). 


Figure I: Assessor Designation and Frequency of 
Mini-CEX Encounters 


Assessor Designation and Number of Times Assessment Done 
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Figure Il: Mean Plot of Student Satisfaction using Table Il: Mean score in OSCE Pre and Post 
MINI CEX Mini-CEX Sessions 
Std. 
on iN Deviation 
Pre-Mini CEX 
_. OSCE marks 60.2273 = 22 8.20529 
oe 
OstT-IMINI 
OSCE marks 64.2273 22 6.68283 
Table Ill: 


Paired Mean Score differences between Pre and 
Post Mini-CEX OSCE 


Mean of Student satisfaction using mini-CEX 


Mean Std. P-value 
Deviation 
Pre Mini CEX 
OSCE Marks - 3.5456 .000 
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Table I: Comparison of Means of Competencies Evaluated during Eight Mini-CEX Sessions 


Jan 21 Feb 21 Mar 21 Apr 21 May 21 Jun 21 July 21 Aug 21 
Skills Assessed tY|YyYyYY____Y__W___"|"____WH#!_]1____1____WH1_W1———— p-value 
Mean+SD Mean+SD MeantSD MeantSD Mean+SD Mean+SD MeantSD Mean+SD 
History Taking 3.864.35 4.184.58 4.54.59 5.044.65 5.22+.81 5.95+.72 6.36+.84 6.86+.88 <.0001 


PhysicalExam 3.904.42 44.61 4.64.65 4,864.71 5.594.85 6.134.63 6.22+.86 7.13+.99 <.0001 


Thifns 3.68+.71 4.13+.83 4.404.59 4.95+.84 5.594.66 5814.95 6.40410 6.94.92 <.0001 
ea 3.72+.45 4.18+.58 4.45+.67 4.90+.68 5.18+.66 5.86+.94 6.36+.72 6.81+1.05 <.0001 
aa 3.54+.73 4.18+.85 4.63+.65 4.90+.97 5.54+.80 6.13+.94 6.54+.73 7.04+.99 <. 0001 


Professionalism 4.0+ .00 4.09+.42 4.45+.59 4.86+.56 5.27+.63 5.68+.89 6.22+.81 6.81+1.0 <. 0001 


Organization/ 


Efficiency 3.59+.79 4.09+.86 4.68+.56 5.27+.82 5.36+.95 5.95+.84 6.72+.88 7.09+1.06 <.0001 


Overall Clinical 


Competence 3.77+.52 4.18+.58 4.68+.64 5.13+.63 5.54+.80 6.0+.69 6.68+.71 7.13+.99 <. 0001 
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DISCUSSION 


Mini-CEX helps assess a trainee's ability to focus on 
diagnosis, management plan, and counselling skills in 
real-life clinical practice using direct Observation’. In 
Pakistan, only a few studies have been conducted at 
undergraduate and postgraduate levels for assessing 
the effectiveness and utility of Mini-CEX as a 
workplace-based evaluation tool in residency training 
programs’. As per our knowledge, this was the first 
local study of its kind to be conducted with Pediatric 
postgraduate residents to assess the effectiveness of 
Mini-CEX in improving their clinical competencies. 
This study observed a significant and progressive 
improvement in the postgraduate — residents' 
performance in nearly all the clinical competencies in 
subsequent Mini-CEX encounters. The difference 
between the mean scores for each clinical skill on the 
first Mini-CEX session and the last session was 
statistically significant. These findings can be 
explained by the fact that Mini-CEX helps identify 
gaps in teaching and learning, and _ instructional 
strategies can be modified accordingly®. Moreover, 
timely, constructive, and contextual feedback given to 
the trainees by the evaluators led to the observed 
improvement in performance. The findings of this 
study agree with the results of Azeem M et al.2and a 
few other studies from India'’®"", wherein the clinical 
performance of most residents significantly improved 
in the later sessions of mini-CEX when juxtaposed 
with the initial session. In a study on Mini-CEX by 
Baqai S 2018'*, a substantial improvement in the 
Clinical performance of postgraduate residents in 
Gynecology and Obstetrics was observed, which they 
attributed to direct Observation and timely and 
constructive feedback. Our study showed profound 
improvement in residents’ organizational efficiency 
and clinical management skills. Similar findings have 
been reported in a study from Lahore by Zareen A 
2018'°, wherein maximum improvement was observed 
in the trainees’ organizational efficiency. However, a 
study from Islamabad by Saeed N 2015" with 
undergraduate trainees reported a profound 
improvement in history-taking, clinical judgment, and 
communication skills over successive Mini-CEX 
encounters. In contrast to other local studies'*"°, a 
different aspect of this study was an OSCE conducted 
before the first Mini-CEX session and after the last 
session, and the scores were compared after that. A 
significant improvement in the mean OSCE score was 
observed, which was indicative of the efficacy of Mini- 
CEX in improving the clinical competencies of 
trainees. A similar study by Kim S et al.” corroborates 
these findings, wherein a decrease in the failure rate 
on the undergraduate summative OSCE was 
observed from 12% pre-Mini-CEX to 2% post-Mini- 
CEX (p-value =0.046). On the contrary, a study from 
Peshawar by Shah MI 2021"° did not find Mini-CEX 
effective in improving undergraduate — students' 
summative assessment scores. 
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To improve reliability and minimize inter-rater bias in 
this study, multiple faculty members conducted 8 Mini- 
CEX encounters per trainee. Moreover, 89.9% of the 
clinical cases in our study were of similar complexity. 
Several studies ‘’®have advocated using multiple 
raters and at least 8 Mini-CEX encounters in an 
academic year of a trainee for better reliability and 
validity. We observed a significant and progressive 
increase in the mean satisfaction level of the faculty 
and residents over subsequent Mini-CEX encounters. 
An increase in the residents’ satisfaction level can be 
explained based on the feedback provided at the end 
of each Mini-CEX was directed at improving the 
trainee's performance and fostering self-reflection. 
The high satisfaction rate of the faculty can be 
attributed to the flexibility and convenience of 
conducting Mini-CEX encounters'®. Goel A 2015”° 
and Gupta S 2017*' have reported similar results, with 
a majority of students feeling satisfied with the 
feedback provided to them by their teachers and 
considering it feasible in most Pediatric clinical 
settings. A similar study by Khalil S 2017” and Bashir 
K et al.“ reported that faculty and residents had high 
acceptability for Mini-CEX. Zareen A 2018'° have 
reported a satisfaction rate with Mini-CEX of 87.5% in 
Gynaecology and Obstetrics postgraduate residents. 
This study's average observation time was 16.5 
minutes, while the feedback time was 9 minutes. 
Saeed N 2015" reported similar results with an 
average observation time of 16.4 minutes and 9.8 
minutes for feedback. Thus, the entire exercise can be 
easily adjusted to the busy work schedule of the 
faculty. 


CONCLUSION 


Mini-CEX as a formative assessment tool significantly 
improves the clinical competencies of Pediatric 
postgraduate residents with subsequent clinical 
encounters. It is recommended that further research 
studies to elucidate the role of Mini-CEX in improving 
clinical learning be conducted in Pakistan to 
reorganize undergraduate and post-graduate national 
curricula. 
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